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PREFACE 


The  current  study  explored  the  operational  utility  of  using  synchronized,  multi-sensor 
images  for  the  acquisition  of  pre-brief  ground  order  of  battle  targets  and  specifically  for 
distinguishing  real  targets  from  decoys.  The  rates  of  target  Hits  and  False  Alarms  with  a 
synthetic  aperture  radar  (SAR)  sensor  alone  were  compared  to  the  corresponding  rates 
when  both  SAR  and  forward  looking  infrared  (FUR)  sensors  were  employed 
sequentially. 

The  study  was  conducted  with  the  joint  participation  of  the  Simulators  and  Human 
Factors  Engineering  Branch  of  the  Israel  Air  Force  (IAF)  and  the  Crew  System  Interface 
Division  of  the  United  States  Air  Force  Research  Laboratory’s  Human  Effectiveness 
Directorate  (AFRL/HEC).  The  research  took  place  in  Israel  during  the  years  2000-2001. 
Mr.  Gilbert  Kuperman  (AFRL/HECA),  Lt  Col  Itzhak  Nadler,  and  Major  Michal  Chovav 
(IAF)  served  as  technical  project  officers. 

This  study  was  carried  out  with  the  contractual  support  of  Synergy  Integration  Ltd.,  Tel 
Aviv,  Israel,  who  prepared  the  simulated  sensor  imagery  and  did  the  programming  for  the 
experiment. 

The  authors  wish  to  thank  the  photo  interpreters  who  volunteered  to  participate  as  subject 
matter  experts  in  the  experiment  and  their  commanding  staff  who  helped  in  organizing 
the  project.  Special  thanks  are  due  to  Major  Chovav  who  coordinated  these  efforts  during 
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SECTION  I 


INTRODUCTION 


Multi-Sensor  Imaging 

The  combined  use  of  various  types  of  sensor  imagery  can  enhance  both  speed  and 
accuracy  of  imagery  interpretation.  Speed  may  be  improved  by  combining  means  that 
allow  fast  screening  of  a  large  scene  with  means  that  enable  thorough  investigation  of 
individual  targets.  Accuracy  of  interpretation  may  be  improved  when  a  combination  of 
sensors  provides  more  information  than  each  of  the  sensors  provides  separately. 
Specifically,  such  a  combination  may  potentially  enhance  the  distinction  between  real 
targets  and  decoys. 


SAR  and  FLIR  Sensors 

If,  for  example,  both  a  SAR  (synthetic  aperture  radar)  sensor  and  an  Electro-Optical  (EO) 
sensor  (e.g.,  FUR,  forward  looking  infrared)  can  be  used  in  the  same  area,  a  user  could 
benefit  from  the  relative  advantages  of  each  of  them: 

SAR  may  provide  images  from  a  long  distance  (standoff  range),  which  are  not  affected  by 
lighting  and  atmospheric  conditions.  However,  SAR  has  a  relatively  limited  resolution 
and  it  provides  a  non-literal  picture  in  which  gray  shades  are  determined  by  the  radar 
reflectivity  of  objects  in  the  scene.  In  comparison,  EO  sensors  may  provide  higher 
resolution  and  a  more  natural  representation  of  the  scene  (depending  on  type  of  sensor). 

In  addition,  some  EO  systems  may  be  much  smaller,  lighter  and  more  available  then  SAR 
systems.  They  may,  therefore,  be  installed  in  larger  numbers  of  smaller  platforms  such  as 
small  UAVs  (uninhabited  air  vehicles),  helicopters,  small  vehicles  and  even  hand  held 
systems.  However,  all  EO  sensors  are  affected  by  atmospheric  conditions  and  by  time  of 
day  (lighting  or  heat  emittance)  and  may  therefore  be  more  limited  in  range. 

Clearly,  SAR  and  EO  sensors  may  complement  each  other  and  their  combined  use  may 
provide  many  advantages.  For  example,  SAR  images  may  be  used  for  detecting 
suspected  objects  at  long  distances  while  EO  sensors  may  then  be  used  for  the  recognition 
stage  of  information  extraction. 

In  order  to  take  advantage  of  multiple  image  sources,  it  is  important  to  understand  the 
parameters  that  determine  the  relative  contribution  of  each  of  the  participating  sensors 
and  to  understand  the  circumstances  under  which  they  may  complement  each  other.  The 
objective  of  the  present  study  is  to  investigate  these  issues. 
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SAR  and  FLIR  signatures 


The  SAR  signature  (proportionate  to  the  RCS,  radar  cross  section)  depends  on  the 
geometry  of  an  object  and  on  its  material  properties.  It  is  not  affected  by  atmospheric 
conditions,  time  of  day,  level  of  activity  (e.g.,  engine  running),  etc.  In  contrast,  IR 
(infrared)  images  are  much  less  consistent  because  they  are  affected  by  all  of  the  above 
variables.  The  IR  signature  of  a  typical  military  target  is  affected  primarily  by  its  heat- 
producing  components  (e.g.,  engine,  treads,  gun,  etc.).  However,  these  components  may 
be  hot  or  cold,  depending  on  whether  and  when  the  target  has  operated  its  engine, 
changed  its  position,  fired  the  gun,  etc. 

As  a  result,  if  an  object  presents  a  SAR  signature  which  lacks  meaningful  detail,  it  may  be 
decided  that  the  object  is  “not  a  target.  ”  The  FUR  signature  however  is  much  more 
"tricky"  and  potentially  deceiving  because  a  cold  object  may  be  an  inactive  target. 


The  Operational  Use  of  Decoys 


An  observer  of  a  military  scene  may  have  difficulty  detecting,  classifying,  recognizing 
and  identifying  all  the  types  of  military  and  non-military  equipment  which  may  be  present 
in  the  scene.  The  task  may  be  even  more  challenging  if  the  adversary  makes  effective 
usage  of  decoys.  Decoys  are  employed  to  provide  false  "targets"  against  which  the  enemy 
will  expand  its  efforts,  possibly  revealing  its  position  in  the  process.  Decoys  also  enhance 
friendly  survivability  and  may  deceive  the  enemy  about  the  number  and  location  of 
friendly  troops  and  equipment. 

A  recent  case  study  (Cohen  &  Shelton,  1999)  analyzed  the  effects  of  decoys  on  NATO’s 
war  in  Kosovo.  NATO  conducted  a  78-day  air  campaign  over  Kosovo.  The  US  Army 
officially  claimed  that  122  tanks  and  more  than  220  troop  carriers  and  other  military 
vehicles  were  destroyed.  However,  when  the  ground  troops  (and  the  media)  entered  the 
area  there  was  hardly  any  evidence  of  destroyed  military  targets.  At  the  same  time  the 
Serb  army  was  seen  to  withdraw  from  Kosovo  with  hundreds  of  intact  tanks,  cannons, 
and  multiple  launch  rocket  systems  (MLRS).  Apparently,  the  bombing  campaign,  which 
consisted  of  more  than  3,000  sorties  and  the  expenditure  of  tens  of  thousands  of  tons  of 
explosives,  including  the  use  of  sophisticated  precision  weapons,  succeeded  in  damaging 
only  about  13  -  30  of  the  Serb’s  300  battle  tanks  in  Kosovo.  The  exact  numbers  depend 
on  the  source;  NATO’s  official  numbers  are  higher  than  most  media  reports  (e.  g., 
Norton-Taylor,  2000). 

According  to  some  media  reports  (e.g.,  Beaver,  1999)  these  outcomes  are  a  result  of  two 
factors:  first,  the  real  equipment  was  hidden  in  various  safe  locations  and  second,  the 
Serbs  used  large  numbers  of  decoys,  smartly  located  to  deceive  pilots  flying  at  an  altitude 
above  15,000  feet.  The  Serbs  used  pneumatic  (i.e.,  inflatable)  rubber  "images"  of  tanks 
(Figure  1),  which  included  a  heat  source  for  decoying  thermal  imaging  systems  (On 
February  18, 2000,  The  Herald  Tribune  wrote  that  many  of  the  heat  sources  were 
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domestic  microwave  ovens  looted  from  Albanian  homes.)  The  decoys  were  cleverly 
located  next  to  phony  bridges  that  were  constructed  on  fake  roads  made  of  black  plastic 
sheets. 

Decoys  can  be  elaborate  or  simple,  pre-constructed  or  made  from  field-expedient 
materials.  The  fidelity  (realism)  of  decoys  depends  on  whom  and  what  sensors  they  are 
meant  to  deceive.  Completely  replicating  all  features  of  real  targets  can  be  very  difficult 
and  expensive,  especially  for  moving  and  mobile  targets.  Hence,  in  most  cases,  decoys 
may  aim  at  deceiving  certain,  but  not  all,  aspects  of  the  target’s  signature  (e.  g.,  visible 
features,  size,  shape,  brightness  and  color,  movement  and  motion,  ground  effects,  thermal 
signature,  RCS,  EM  [electro-magnetic]  signature).  As  a  result,  observing  a  target  by 
different  means,  which  exploit  different  phenomenologies,  may  enhance  the  ability  to 
distinguish  decoys  from  real  targets.  For  example,  had  NATO’s  pilots  used  SAR,  they 
might  not  have  fallen  for  the  pneumatic  rubber  tank  decoys,  for  the  wooden  bridges  and 
for  the  plastic-sheet  roads. 


Figure  1  -  A  destroyed  pneumatic  tank  decoy,  next  to  a  decoy  bridge  in  Kosovo 
(from  BBC  News  Online  -  June  25, 1999) 


The  recognition  of  targets  and  decoys 


Decoys  may  differ  from  targets  in  any  of  a  number  of  features  (e.g.,  size,  shape,  texture, 
motion  and  movement  capabilities,  RCS,  thermal  signature,  etc.).  In  order  to  distinguish 
targets  from  decoys,  an  observer  must  be  able  to  recognize  some  of  these  differences. 

The  distinction  between  targets  and  decoys  is  a  specific  case  of  object  recognition  that 
can  be  examined  in  view  of  psychological  object  recognition  theories.  Most  relevant  in 
the  present  context  are  Feature  Theories.  Feature  theories  postulate  that  the  visual  system 
analyses  and  represents  sensory  information  in  abstract,  primitive  information  units  called 
features  or  attributes;  a  “distinctive  feature”  can  be  used  to  make  a  critical  distinction 
between  patterns  or  classes  of  objects  (e.g.,  Treisman  and  Gelade,  1980).  The  recognition 
of  a  pattern  involves  the  analysis  of  its  features  (e.g.,  line  direction,  size,  color,  etc.). 
Feature  theories  received  strong  reinforcement  from  neurological  studies,  which 
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identified  brain  cells  sensitive  to  specific  features  (e.g.,  lines  of  a  specific  orientation, 
comers  and  angles  -  Hubei  and  Wiesel,  1977).  Biederman  (1987)  postulated  a  means  by 
which  stable,  three-dimensional  mental  representation  of  objects,  can  be  based  on  spatial 
arrangements  of  simple  geometric  shapes  he  called  “geons."  According  to  Biederman’s 
Recognition-by-Component  theory,  objects  are  recognized  by  observing  and  extracting 
their  edges,  and  then  decomposing  them  into  geons,  which  can  then  be  recomposed  into 
alternative  arrangements. 

Feature  theories  seem  most  relevant  to  the  present  context.  Decoys  are  purposely 
designed  to  resemble  targets.  However,  they  may  differ  from  targets  in  some  of  their 
features.  The  role  of  the  observer  is  to  detect  and  recognize  these  distinguishing  features. 
In  the  present  study,  the  observers  are  asked  to  decide  whether  an  object  is  a  target  or  “not 
a  target”  (rather  then  positively  recognizing  the  decoys).  This  decision  can  be  reached  in 
either  a  positive  or  a  negative  approach: 

1.  The  positive  approach  is  to  determine  that  an  object  has  all  the  necessary 
(positive)  features  and  only  the  positive  features  that  characterize  a  target. 

2.  The  negative  approach  is  to  recognize  at  least  one  negative  feature  that  precludes 
the  object  from  being  a  target. 

The  positive  approach  is  almost  inevitably  longer  and  more  tedious  than  the  negative 
approach  since  it  is  serial  and  exhaustive.  Hence,  an  observer  may  spontaneously  look 
for  negative  features  rather  than  try  to  sum  up  all  positive  features.  Obviously,  the  chance 
of  recognizing  negative  features  (that  precludes  the  object  from  being  a  target)  is  higher  if 
there  are  several  such  features,  and  if  the  features  are  salient.  By  potentially  revealing  a 
larger  variety  of  features,  the  multi-sensor  approach  has  two  potential  advantages:  First, 
there  is  a  better  chance  for  the  presence  (and  recognition)  of  negative  features.  Second,  a 
combination  of  many  features  may  provide  better  and  more  conclusive  distinction 
between  a  target  and  a  decoy. 


Objectives 


The  objectives  of  the  present  study  are: 

1 .  to  investigate  working  procedures  of  imagery  analysts  using  S  AR  and  FOR 
representations  of  the  same  area 

and 

2.  to  identify  the  effects  of  major  image  parameters  on  the  imagery  interpretation 
process. 
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An  image  library  of  the  same  area  was  represented  in  simulated  S  AR  and  simulated  FUR 
sensor  imagery.  The  simulated  terrain  was  created  from  a  real,  three-dimensional  terrain 
database  (extracted  from  the  Ramat  Hagolan  area).  Variations  of  the  background  scene 
were  created  by  adding  two  and  three-dimensional  objects  and  targets  (e.g.,  roads,  trees, 
buildings,  military  targets  and  decoys). 

During  each  trial  of  the  experiment,  participants  were  asked  to  designate  all  instances  of  a 
pre-specified  target  in  the  S  AR  images  that  contained  both  targets  and  decoys. 

Participants  were  asked  to  designate  all  targets,  and  to  avoid  the  designation  of  decoys. 
After  the  designation  of  each  object,  participants  were  asked  to  indicate  their  level  of 
confidence  that  the  object  was  a  target  (on  a  1  -  7  scale).  After  the  designation  of  all 
suspected  targets  (or  a  maximum  allowed  period  of  time  with  the  SAR  image), 
participants  were  able  to  designate  an  object  in  the  SAR  image  and  then  view  a  FLIR 
image  of  the  designated  target  location.  They  could  then  confirm  or  modify  their  former 
target  /  decoy  decision,  and  update  their  level  of  confidence  for  that  object. 

It  is  hypothesized  that  the  use  of  both  SAR  and  FUR  imagery  will  increase  participants’ 
ability  to  distinguish  targets  from  decoys,  relative  to  the  use  of  SAR  imagery  alone. 
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SECTION  n 


METHOD 


Subjects 

Eight  Subject  Matter  Experts  (SMEs),  seven  male  and  one  female  imagery  analyst  of  the 
Israel  Air  Force,  participated  in  the  experiment.  The  SMEs  had  0.5  to  4.5  years  of 
operational  experience  in  image  interpretation.  The  main  expertise  of  all  SME  was  in  the 
interpretation  of  regular  (literal,  photographic  or  electro-optical  [television])  imagery.  All 
SMEs  had  some  SAR  interpretation  experience.  Two  had  some  familiarity  with  FUR 
imagery  while  the  other  six  had  no  experience  with  exploiting  FUR.  All  SMEs  had 
normal  or  corrected  to  normal  vision. 


Test  materials 


Terrain 

Three  different  areas  were  created,  based  on  one  sampled  digital  terrain  elevation  data 
base  (of  the  Ramat  Hagolan  area),  by  inserting  different  "overlays"  of  two  and  three- 
dimensional  objects  to  the  sampled  terrain.  The  added  objects  included  roads,  vegetation 
(trees)  and  buildings  (Figures  2  and  3). 

Objects:  targets  and  decovs 

Targets 

Six  ground  order  of  battle  vehicles  were  used  as  targets  in  the  experiment:  M-l  main 
battle  tank,  T-72  main  battle  tank,  MLRS,  air  defense  radar,  SCUD 
transporter/erector/launcher,  and  SA-6  surface-to-air  missile  launcher  (Appendix  1).  The 
number  of  targets  of  each  type  in  each  trial  was  between  1  and  6,  depending  on  type  of 
target  (1  or  2  SCUD;  3  SA-6;  4  MLRS;  2  or  3  Radar;  5  or  6  Tanks).  These  numbers  were 
selected  in  order  to  create  realistic  operational  scenarios.  All  targets  were  entered  as 
computer-aided  design  (CAD)  models  in  the  sensor  simulation  applications  (see  below). 

Decovs 

Four  types  of  decoy  objects  were  defined.  Three  of  them  were  derivatives  of  the  "real" 
target  and  the  fourth  was  one  of  the  "other"  targets: 

A  Similar-decoy  (i.e.,  derived  from  the  corresponding  target  CAD  model)  was  created  for 
each  target  by  introducing  small  variations  in  the  target’s  physical  appearance  (Appendix 
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1).  These  were  briefed  to  the  SMEs  as  “real  objects”  that  are  decoys  for  the  purpose  of  the 
experimental  task  (e.  g.,  low  priority  targets,  high  fidelity  decoys,  etc.). 

A  SAR-decoy  was  created  for  each  target,  by  introducing  small  structural  modifications  to 
the  CAD  model  (e.  g.,  by  removing  small  parts  such  as  antennas,  etc.).  In  order  to  create  a 
realistic  decoy  against  the  S AR  sensor,  the  SAR-decoy  had  a  similar  material  definition  to 
that  of  the  actual  targets.  The  thermal  (FUR)  signature  of  SAR-decoys  was  similar  to  that 
of  a  cold  target. 

A  FLIR-decoy  object  was  created  for  each  target  by  introducing  small  structural  changes 
(e.  g.,  removing  small  parts  such  as  antennas,  etc.).  In  order  to  provide  a  realistic  decoy 
against  the  FUR  sensor,  the  FLIR-decoy  had  a  thermal  signature  that  was  similar  to  the 
signature  of  an  actual,  hot  target.  The  FLJR-decoys  were  described  as  being  fabricated 
from  "soft"  materials  (e.  g.,  wood  or  plastic),  and  were  therefore  only  barely  visible  by  the 
SAR. 

During  each  trial,  a  single  target  was  pre-briefed  to  the  SME  as  the  designated  target  for 
that  trial.  Thus,  other  target  objects  could  be  used  as  decoys.  These  objects  are  called 
Target-decoys.  In  practice,  only  one  type  of  target  and  one  Target-decoy  (and  the  other 
types  of  decoys)  were  presented  during  one  trial.  Each  of  the  targets  was  assigned  a 
counterpart  target  that  served  as  its  Target-decoy.  The  pairs  were: 

•  M-l  tank  and  T-72  tank 

•  Radar  and  MLRS 

•  SCUD  launcher  and  SA-6  launcher 

These  pairs  were  formed  on  the  basis  of  relative  physical  similarity.  In  reality,  however, 
the  degree  of  physical  similarity  differed  between  the  three  pairs:  the  two  tanks  were 
highly  similar,  the  Radar  and  MLRS  were  moderately  similar,  while  the  two  launchers 
were  least  similar. 

All  decoys  were  entered  as  computer-aided  design  models  in  the  sensor  simulation 
applications  (see  below). 

In  addition  to  the  above.  Terrain-objects  (e.g.,  buildings)  could  also  be  confused  with 
targets. 

Relevant  parts  of  targets  and  decoys  (i.  e.,  engines,  wheels,  thread,  guns)  could  be  hot  or 
cold.  FUR-decoys  always  had  a  hot  part.  SAR-decoys’  parts  were  always  cold. 

SAR  imagery 

MultiGen-Paradigm  (http://www.multigen.eom/l  SensorVision™,  SensorWorks™,  and 
RadarWorks™  applications  were  used  to  create  the  SAR  imagery.  The  resolution  of  the 
SAR  was  1  m  per  pixel.  Each  SAR  image  (i.e.,  one  experimental  trial)  represented  an  area 
of  approximately  1000  x  1000  m.,  displayed  at  a  high  grazing  angle  of  30  degrees.  The 
image  was  displayed  with  the  shadows  facing  upwards  (Figure  2). 
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Figure  2  -  Simulated  SAR  imagery:  TOP:  overall  scene  at  reduced  resolution; 
BOTTOM:  scene  extract  at  full  resolution. 
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FLIR  Imagery 


Simulated  FUR  imagery  (Figure  3)  was  generated  from  the  same  terrain  and  target  (or 
decoy)  data  bases  using  MultiGen-Paradigm  SensorWorks  and  SensorVision  applications 
to  create  and  display  the  images.  The  resolution  of  the  simulated  FUR  was  approximately 
twice  the  resolution  of  the  SAR,  i.e.,  approximately  2  pixels  per  1  meter.  The  simulated 
FUR  field  of  view  (FOV)  was  approximately  1.4  degrees,  covering  an  area  of 
approximately  125  x  125  meters  from  a  slant  range  distance  of  10,000  meters.  Depression 
angle  of  the  sensor  was  -45  degrees.  The  magnification  of  the  FUR  was  approximately 
twice  the  magnification  of  the  SAR. 


Figure  3  -  Simulated  FLIR  imagery 


Apparatus 

A  graphics  workstation  equipped  with  a  17”  diagonal,  1280  X  1024  pixel  resolution, 
color  monitor  and  standard  keyboard  and  mouse  were  used  for  stimulus  presentation  and 
data  collection. 
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Experimental  Design  and  Procedure 

Training  phase 

Each  experiment  started  with  a  training  phase,  in  which  the  participants  were  familiarized 
with  the  targets,  with  the  various  types  of  decoy  objects,  and  with  the  experimental 
procedure.  In  addition,  since  most  of  the  SMEs  had  no  FLIR  experience,  they  were 
briefed  about  major  FLIR  imagery  attributes  (see  Appendix  2  for  the  written  instructions). 
The  SMEs  were  shown  images  that  displayed  each  object  from  five  different  viewing 
angles  (top,  front,  back,  left,  and  right)  (Appendix  1).  They  then  performed  two 
procedural  practice  trials  in  which  all  the  targets  and  all  the  decoys  were  arrayed  on  the 
terrain  in  sorted  rows.  This  enabled  the  SME  to  view  and  compare  all  objects  in  their 
simulated  setting  and  to  practice  the  experimental  procedure  (Figure  4).  Following  the 
procedural  trials,  two  additional  trials  (with  the  targets  appearing  as  they  would  in  the 
data  collection  images)  were  performed  for  additional  practice  (Table  1).  The  training 
phase  was  self-paced,  and  lasted  from  45  to  75  minutes. 


Figure  4  -  SAR  Imagery  of  all  targets  and  decoys  (ordered  in  columns  from  left  to  right 
are:  SCUD,  SA-6,  Radar,  MLRS,  M-l,  and  T-72;  ordered  in  rows  from 
bottom  to  top  are:  Targets,  Similar-decoys,  FLIR-decoys,  and  SAR-decoys) 

Experimental  phase 

The  experimental  phase  included  10  trials.  A  within-subject  design  was  used  in  which 
each  participating  SME  performed  the  same  set  of  trials.  The  order  of  the  experimental 
trials  was  reversed  between  participants  to  counterbalance  order  effects.  Table  1  presents 
the  experimental  design. 


Table  1  -  Experimental  design 
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Procedure 

Each  experimental  trial  presented  one  of  the  terrain  backgrounds  with  a  set  of  targets  and 
decoys  distributed  across  it.  A  pre-brief  was  provided  for  each  trial  during  which  the  set 
of  objects  was  presented.  Each  set  contained  the  pre-defined  target,  its  Similar,  S  AR  and 
FLJR-decoys  and  its  Target-decoy.  (See  Table  1  for  the  number  of  objects  in  each 
category  represented  in  each  trial.)  SMEs  were  asked  to  detect  and  recognize  the  pre¬ 
defined  target  and  ignore  decoys. 

The  target  was  designated  by  placing  the  mouse-driven  cursor  over  the  target  and  then 
depressing  the  left  mouse-button.  A  square  red  frame  appeared  around  the  designated 
object  (target,  decoy  or  terrain  object).  Each  trial  consisted  of  two  Phases:  the  SAR-only 
Phase  and  the  SAR&FLIR  Phase.  The  SAR-only  Phase  had  to  be  completed  before  the 
SME  could  move  to  the  SAR&FLIR  Phase.  (See  below  for  more  details). 

(Note:  the  rigid  one-way  process  that  starts  with  the  SAR-only  and  then  moves  to  the 
SAR&FLIR  phase,  was  selected  for  experimental  purposes.  This  process  enabled  us  to 
separately  measure  the  contribution  of  SAR-only  and  of  SAR&FLIR.  Operationally, 
however,  it  would  make  more  sense  to  allow  for  more  flexibility,  e.g.,  command  the  FUR 
imagery  at  any  desired  stage  [assuming  that  the  target  was  in  range].) 

After  the  designation  of  an  object  (in  either  Phase),  the  SME  was  asked  to  specify  his/her 
level  of  confidence  on  a  1  -  7  scale,  by  pressing  one  of  the  F(unction)  keys  on  the 
keyboard  as  indicated  below.  The  red  box  then  changed  color  to  blue. 

The  level  of  confidence  scale 

The  level  of  confidence  measure  represents  the  SME’s  subjective  evaluation  of  the 
attained  level  of  information  extraction.  It  was  defined  as  follows: 
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FI  -  Detection:  The  object  may  be  some  type  of  target. 

F2  -  Recognition:  The  object  is  possibly  a  target  of  the  specified  category  (e.g.,  a 
tank) 

F3  -  Recognition:  The  object  is  probably  a  target  of  the  specified  category 
F4  -  Recognition:  The  object  is  definitely  a  target  of  the  specified  category 
F5  -  Identification:  The  object  is  possibly  the  designated  target  (e.g.,  T-72  tank) 
F6  -  Identification:  The  object  is  probably  the  designated  target 
F7  -  Identification:  The  object  is  definitely  the  designated  target 
F8  -  Not  a  Target:  This  key  was  used  only  during  the  SAR&FUR  Phase.  It 
canceled  the  former  selection  of  a  target. 

The  SAR-onlv  Phase 

Each  trial  started  with  the  SAR-only  Phase  in  which  the  SME  viewed  only  the  SAR 
image  and  was  given  time  to  designate  any  candidate  object.  The  bounding  square  red 
frame  indicated  each  designated  object.  Following  each  designation,  the  SME  rated 
his/her  confidence  level  in  that  designation  by  selecting  an  associated  level  of  confidence. 

The  SAR-only  Phase  lasted  until  the  SME  completed  the  designation  of  all  suspected 
targets  or  until  15  minutes  elapsed  (whichever  happened  first).  During  the  SAR-only 
Phase  the  SME  could  revise  previous  confidence  levels.  The  SME  could  also  "de- 
designate"  an  object  by  assigning  it  to  the  "Not  a  Target"  category  (F8). 

The  SAR&FLTR  Phase. 


After  designating  all  suspected  targets  from  the  SAR  image  (or  after  15  minutes  had 
elapsed),  the  SME  moved  to  the  SAR&FUR  Phase.  The  SME  could  now  select  each  of 
the  previously  designated  SAR  objects,  thereby  opening  a  FUR  image  for  the  area 
containing  that  object.  On  the  SAR  image,  the  frame  surrounding  the  target  turned  green, 
indicating  that  a  FUR  image  had  been  commanded.  The  FUR  image  was  displayed  next 
to  the  designated  SAR  target,  without  concealing  it,  thereby  enabling  the  SME  to  compare 
the  SAR  and  FUR  representations  of  each  object  (Figure  5).  The  designated  object 
appeared  randomly  within  a  radius  of  80  meters  from  the  center  of  the  FUR  image 
(reflecting  a  certain  level  of  possible  misalignment  between  SAR  and  FUR  images). 
Therefore,  the  SME  had  to  locate  the  previously  designated  target,  select  it  on  the  FUR 
and  then  confirm  or  change  the  former  target  /  decoy  decision  and  update  the  confidence 
level  for  that  object.  On  the  displayed  FUR  image,  the  SME  could  designate  any  object, 
even  one  not  previously  designated  on  SAR.  After  the  completion  of  the  rating  procedure 
the  FUR  window  was  closed  by  the  SME.  On  the  SAR  image,  the  frame  surrounding  the 
target  now  turned  white  indicating  that  the  object  had  been  rated  using  a  FUR  image.  A 
new  FUR  window  was  displayed  when  the  SME  selected  another  designated  object  on 
the  SAR.  The  SME  could  not  return  to  previously  displayed  FUR  windows.  Time  for  the 
SAR&FUR  Phase  was  unlimited.  The  trial  ended  when  the  SME  decided  that  the  process 
was  completed,  usually  after  going  through  all  objects  that  had  been  designated  on  the 
SAR  image. 


12 


All  events  of  each  trial  were  recorded  in  a  log  file. 


Figure  5  -  FUR  imagery  displayed  above  the  SAR  imagery  (the  selected 
object  is  a  SCUD  target) 


Dependent  variables 

During  the  experiment,  the  following  dependant  variables  were  recorded: 

1.  Hits:  the  number  of  correct  designations  (designation  of  targets). 

2.  False  alarms  (FAs):  the  number  of  false  designations  (designation  of  decoys  and 
terrain-objects). 

3.  Level  of  confidence:  The  level  of  confidence  as  expressed  by  the  SME  during  each 
Phase. 

Throughout  the  experiment,  accuracy  was  emphasized  over  speed. 
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Section  III 


Results  and  Discussion 


General 

The  major  hypothesis  was  that  the  overall  levels  of  information  extraction/target 
designation  performance  would  be  better  in  the  SAR&FLER  Phase  than  they  were  in  the 
SAR-only  Phase.  Based  on  the  dependent  measures,  the  following  combined  scores  were 
calculated. 

Hits  and  False  Alarms 


•  Hit  and  FA  Scores:  Four  basic  scores  were  calculated  by  summing  up  the  Hits  and 
the  False  Alarms  (FA)  of  every  trial  of  each  SME  for  each  Phase  (i.e.,  SAR-only 
and  SAR&FLIR  Phases). 

•  Mean  Hit  and  FA  Rates:  The  Hit  and  FA  Scores  were  transformed  into  rates  by 
dividing  the  actual  scores  by  the  maximum  scores,  where  maximum  Hits  is  the 
total  number  of  targets,  and  maximum  FA  is  the  total  number  of  decoy  objects 
(not  including  terrain-objects).  The  rates  were  then  averaged  across  SME  and 
across  trials,  for  each  Phase  (i.e.,  SAR-only  and  SAR&FLIR  Phases). 

•  Difference  Scores:  Difference  Scores  were  calculated  for  each  SME  by 
subtracting  the  mean  Hit  and  FA  Rates  in  the  SAR&FLER  Phase  from  the  mean 
Hit  and  FA  Rates  (respectively)  in  the  SAR-only  Phase. 

•  Productivity  Scores:  Productivity  Scores  were  calculated  for  each  participant  at 
each  Phase  by  dividing  mean  Hit  Rates  by  the  mean  FA  Rates,  for  the  SAR-only 
and  SAR&FLIR  Phases. 


An  additional  calculation  was  preformed  to  compute  a  Designation  Rate  It  was  the  sum 
of  Hits  and  FAs  divided  by  the  total  number  objects  present  in  each  trail.  It  was  pooled 
over  trials  and  SMEs. 


Table  2  summarizes  the  resulting  data: 


Table  2  -  Mean  Hit  and  FA  Rates,  Productivity  Scores  and  Difference  Scores 

SAR-only  SAR&FLIR  Difference 
Phase  Phase 


Mean  Hit  Rate 

Mean  False  Alarm  Rate 

Productivity  Score  (Hit  Rate/FA  Rate) 


Mean 

(Standard  Error) 

Mean 

(Standard  Error) 

Mean 

(Standard  Error) 

80.08 

65.46 

14.63 

(4.48) 

(7.30) 

(3.90) 

70.66 

37.72 

32.94 

(5.68) 

(4.61) 

(4.60) 

1.20 

2.31 

(0.05) 

(0.18) 

Both  the  mean  Hit  and  FA  Rates  were  higher  during  the  SAR-only  Phase  (MHit=80.08, 
SeHit=4.48;  MFa=70.66,  SeFA=5.68)  than  during  the  SAR&FLIR  Phase  (Mhif65.46, 
SeHit=7.30;  MFA=37.72,  SeFA=4.61)  (Figure  6).  An  analysis  of  variance,  in  which  one 
factor  was  Phase  (SAR-only  or  SAR&FLIR)  and  the  other  was  Designation  Status  (Hit  or 
FA),  revealed  a  significant  main  effect  for  Phase  (F( 1,28)= 17. 84,  p<0.001).  Overall 
Designation  Rates  were  higher  in  the  SAR-only  Phase  (M=75.37,  Se=3.70)  than  in  the 
SAR&FLIR  Phase  (M=51.59,  Se=5.50).  A  main  effect  was  also  found  for  Designation 
Status  (F(l,28)=10.89,  p<0.005).  Averaging  across  the  two  Phases,  Hit  Rates  (M=72.77, 
Se=4.55)  were  higher  than  FA  Rates  (M=54.19,  Se=5.53).  The  interaction  between  Phase 
and  Designation  Status  was  nearly  significant  (F(l,28)=2.65,  p=0.058,  one-tailed).  When 
shifting  from  the  SAR-only  Phase  to  the  SAR&FLIR  Phase,  the  FA  Rate  decreased  more 
rapidly  than  did  the  Hit  Rate. 


The  Difference  Score  for  Hits  (M=14.63,  Se=3.90)  was  lower  than  the  Difference  Score 
for  FA  (M=32.94,  Se=4.60)  (t(7)=4.52,  p<0.005)  (Figure  7). 
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Designation  Status 


Figure  7  -  Difference  Scores  for  Hits  and  False  Alarms 


The  Productivity  Score  was  higher  in  the  SAR&FUR  Phase  (M=2.31,  Se=0.18)  than  in 
the  SAR-only  Phase  (M=1.20,  Se=0.05)  (t(7)=5.62,  p<0.001)  (Figure  8) 


Figure  8  -  Productivity  Scores  by  Phase 


During  the  SAR-only  Phase,  the  SMEs  produced  high  levels  of  both  Hits  and  FAs. 
Approximately  80  percent  of  the  targets  and  70  percent  of  decoys  were  designated  (Table 
2  and  Figure  6).  This  was  not  surprising  given  that  the  SMEs  understood  that  it  was 
beneficial  to  their  overall  performance  to  make  low-confidence  selections  during  the  first 
Phase  (see  Confidence  Ratings  below).  They  realized  that  during  the  second  Phase 
(SAR&FLIR)  they  could  eliminate  FAs  but  had  no  way  to  compensate  for  misses 
(because  only  those  objects  that  were  designated  during  the  SAR-only  Phase  could  be 
examined  during  the  SAR&FUR  Phase).  In  other  words,  during  the  SAR-only  Phase  the 
SME  employed  a  “liberal”  criterion  and  designated  almost  every  suspected  object. 
Nonetheless,  approximately  20  percent  of  the  targets  were  missed,  this  may  be  due  to 
inability  to  detect  the  targets  (even  though  SME  had  almost  unlimited  time  to  scan  each 
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scene)  or  due  to  incorrect  object  rejection  decisions.  The  fact  that  FA  Rate  was  only 
moderately  smaller  than  the  Hit  Rate  provides  some  indication  to  the  difficulty  of  the 
task.  Only  around  30  percent  of  the  decoys  were  rejected  with  full  confidence  during  the 
SAR-only  Phase. 

During  the  SAR&FUR  Phase,  both  Hits  and  FAs  decreased  significantly.  However,  Hit 
Rates  decreased  by  approximately  14.6  percent  whereas  the  FA  Rate  decreased  by 
approximately  33  percent  (Table  2  and  Figure  7)  producing  significant  Difference  Scores. 
These  changes  indicate  that  the  SAR&FUR  provided  SME  with  higher  sensitivity  to 
differences  between  targets  and  decoys  than  the  SAR-only.  However,  it  also  indicates  a 
change  in  their  decision  criterion.  During  the  SAR&FUR  Phase,  the  SMEs  were  more 
conservative  than  during  the  SAR-only  Phase  and  while  decreasing  their  FA  Rate  they 
also  had  to  sacrifice  some  Hits.  The  overall  task  remained  difficult  and  in  order  to 
produce  approximately  65  percent  Hits,  the  SMEs  also  produced  approximately  37 
percent  FAs. 

The  Productivity  Score  (Figure  8)  provides  an  additional  view  of  the  data,  showing  that 
overall  productivity  was  significantly  higher  during  the  SAR&FUR  Phase  than  during  the 
SAR-only  Phase. 


Confidence  Ratings 

Four  basic  Confidence  Scores  were  calculated  for  each  SME  by  averaging  the  Confidence 
Ratings  for  Hits  and  FA  in  each  trial  within  Phase  (SAR-only  and  SAR&FUR). 

Two  Difference  Scores  were  calculated  for  each  participant  by  subtracting  the  mean  Hit 
and  mean  FA  Confidence  Scores  in  the  SAR-Only  Phase  from  the  Hit  and  FA  Confidence 
Scores  in  the  SAR&FUR  Phase  (Table  3). 


Table  3  -  Mean  Confidence  Ratings 


SAR-only 

SAR&FUR 

Difference 

Phase 

Phase 

Mean 

Mean 

Mean 

(Standard  Error) 

(Standard  Error) 

(Standard  Error) 

Hits 

3.11 

5.57 

2.46 

(0.55) 

(0.31) 

(0.31) 

False  Alarms 

2.94 

4.95 

2.01 

(0.50) 

(0.32) 

(0.30) 

Both  mean  Hit  and  mean  FA  Confidence  Scores  were  higher  in  the  SAR&FUR  Phase 
(Mnit=5.57,  SeHit=0.31;  MFa=4.95,  SeFA=0.32)  than  in  the  SAR-only  Phase  (MHit=3.11, 
SeniFO.55;  MFA=2.94,  SeFA=0.50)  for  both  Hits  and  FAs.  (Figure  9).  In  an  analysis  of 
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variance,  in  which  one  factor  was  Phase  and  the  other  was  Designation  Status,  a  main 
effect  was  found  for  Phase  (F(l,28)=26.80,  p<0.001).  Overall  Confidence  Scores  were 
higher  in  the  SAR&FUR  Phase  (M=5.26,  Se=0.27)  than  in  the  SAR-only  Phase  (M=3.03, 
Se=0.36).  No  other  effects  were  significant  in  this  analysis. 


Figure  9  -  Confidence  Ratings  for  Hits  and  FA  by  Phase 


The  Difference  Score  for  Confidence  in  Hits  (M=2.46,  Se=0.31)  was  higher  than  the 
Difference  Score  for  FAs  (M=2.01,  Se=0.30)  (t(7)=2.01,  p<0.05,  one-tailed)  (Figure  10). 


Figure  10  -  Confidence  Ratings  Difference  Scores  for  Hits  and  FA  (pooled 
across  Phase  and  SME) 

The  Confidence  Ratings  during  the  SAR&FUR  Phase  were  significantly  higher  than 
during  the  SAR-only  Phase.  This  was  anticipated  because  during  the  SAR-only  phase,  the 
SMEs  were  expected  to  designate  objects  even  if  their  confidence  level  was  very  low, 
whereas  during  the  SAR&FUR  Phase  they  were  expected  to  make  a  final  decision,  based 
on  reasonably  high  confidence  levels.  As  expected,  the  Confidence  Ratings  for  Hits  were 
significantly  higher  than  for  FAs.  Similarly  and  as  had  been  found  for  the  Difference 
Scores,  the  Confidence  Ratings  for  targets  during  the  SAR&FUR  Phase  increased  more 
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than  the  Confidence  Ratings  produced  for  designated  decoys.  However,  the  differences 
between  them  were  rather  small  during  both  Phases  (Table  3  and  Figure  10).  This  may  be 
an  additional  indication  to  the  difficulties  that  SME  encountered  in  distinguishing 
between  targets  and  decoys.  It  is  also  possible  that  the  SMEs  decided  that  if  they  repeated 
the  same  decision  (in  both  the  SAR-only  and  in  the  SAR&FLJR  phases),  it  must  reflect  a 
higher  level  of  confidence,  even  if  there  was  no  real  ground  for  such  confidence. 

False  Alarms  for  various  decoys 

During  each  trial,  one  target  was  identified  as  the  designated  target  for  that  trial  and  the 
other  objects  were  the  various  decoys  of  that  target.  Each  target  had  four  types  of  decoys, 
three  of  which  were  derivatives  of  the  target.  The  fourth  decoy  (the  “Target-decoy”)  was 
its  counterpart  among  the  other  targets. 

Corrected  False  Alarm  Scores 

To  calculate  the  distribution  of  FAs  per  type  of  decoy,  Corrected  FA  Scores  were 
calculated.  The  rate  of  FA  per  type  of  decoy  was  corrected  by  the  frequency  of  occurrence 
of  that  decoy.  The  calculation  was  performed  as  follows: 

The  number  of  FAs,  per  type  of  decoy,  per  trial  of  each  SME,  and  for  each  Phase,  was 
divided  by  the  total  number  of  objects  of  that  type.  For  example,  if  SME  #1  designated  2 
SAR-decoys  as  targets  and  the  total  number  of  SAR-decoys  in  that  trial  was  3,  then  the 
score  was  2/3=0.66.  These  scores  were  then  averaged  across  trials  and  SME.  Figure  11 
and  Table  4  present  the  distributions  of  mean  Corrected  FA  Scores,  by  Phase. 


Similar 
Different 
SAR  Dummy 
— FUR  Dummy 
— 38 — Terrain 


Figure  11  -  Corrected  False  Alarm  Scores  per  decoy  type,  by  Phase 
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Table  4  -  Corrected  False  Alarm  Scores  per  decoy  type,  by  Phase 


SAR-only  Phase 
Mean 

(Standard  Error) 

SAR&FUR  Phase 
Mean 

(Standard  Error) 

Similar-decoy 

0.82 

0.48 

(0.04) 

(0.06) 

Target-decoy 

0.71 

0.35 

(0.07) 

(0.07) 

SAR-decoy 

0.84 

0.38 

(0.04) 

(0.07) 

FUR-decoy 

0.09 

0.20 

(0.02) 

(0.04) 

Terrain-objects 

0.30 

0.06 

(0.11) 

(0.03) 

In  an  analysis  of  variance,  in  which  one  factor  was  Phase  and  the  other  was  Decoy  Type 
(Similar  /  Target  /  SAR  /  FUR  /  Terrain-object),  a  main  effect  was  found  for  Phase 
(F(l,70)=45.96,  p<0.001).  Averaging  across  Decoy  Types,  Corrected  FA  Scores  during 
the  SAR-only  Phase  (M=0.55,  Se=0.05)  were  higher  than  Corrected  FA  Scores  during 
SAR&FUR  Phase  (M=0.30,  Se=0.04).  A  main  effect  was  found  for  Decoy  Type 
(F(4,70)=32.21,  p<0.001).  Averaging  across  the  two  Phases,  Similar-decoy  Corrected  FA 
Scores  (M=0.65,  Se=0.05)  were  higher  than  Target-decoy  Corrected  FA  Scores  (M=0.53, 
Se=0.07)  (t(7)=3.12,  p<0.05).  Similar,  Target  and  SAR-decoy  Corrected  FA  Scores 
(M=0.65,  Se=0.05;  M=0.53,  Se=0.07;  M=0.61,  Se=0.04,  respectively)  were  higher  than 
FUR  and  Terrain-object  decoy  Corrected  FA  Scores  (M=0.15,  Se=0.03;  M=0.18, 

Se=0.06,  respectively)  ((tSimiiar>FUR(7)=13.9,  p<0.001);  (tTarget,FLiR(7)=8.18,  p<0.001); 
(tSAR,HJR(7)=14.00,  P <0.001);  (tsimilar,Terrain(7)=:6.77 ,  p<0.001);  (tTarget,Terrain(7)=4.67, 
p<0.005);  (tsAR,Tetiain(7)=8.05,  p<0.001)). 

An  interaction  was  found  between  Phase  and  Decoy  Type  (F(4,70)=6.54,  p<0.001). 
Similar,  Target,  SAR,  and  Terrain-object  decoy  Corrected  FA  Scores  were  lower  in  the 
SAR&FUR  Phase  (M=0.48,  Se=0.06;  M=0.35,  Se=0.07;  M=0.38,  Se=0.07;  M=0.06, 
Se=0.03,  respectively)  than  in  the  SAR-only  Phase  (M=0.82,  Se=0.04;  M=0.71,  Se=0.07; 
M=0.84,  Se=0.04;  M=0.30,  Se=0.11,  respectively).  In  contrast,  the  FUR-decoy  Corrected 
FA  Scores  were  higher  in  the  SAR&FUR  Phase  (M=0.20,  Se=0.04)  than  in  the  SAR-only 
Phase  (M=0.09,  Se=0.02). 

Overall,  the  different  types  of  decoys  behaved  as  expected.  All  decoys  except  the  FUR- 
decoy  produced  higher  FA  Rates  during  the  SAR-only  Phase  then  during  the  SAR&FUR 
Phase.  This  is  in  line  with  the  previously  observed  general  shift  in  criterion  between  the 
two  Phases.  The  FUR-decoys  produced  fewer  FA  during  the  SAR-only  Phase  because 
they  were  barely  visible  in  the  SAR  imagery.  As  a  result,  they  were  not  designated  during 
this  Phase  and  could  not  be  investigated  during  the  SAR&FUR  Phase.  Some  FUR- 
decoys  were  visible  and  could  be  designated  on  the  FUR  when  they  were  next  to  other 
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SAR  objects  that  were  designated  by  the  SME.  As  a  result,  the  FA  Rate  for  these  decoys 
increased  during  the  SAR&FLER  Phase,  but  remained  relatively  low,  even  though  they 
were  designed  to  deceive  the  FUR.  FA  Rates  of  SAR-decoys  dropped  to  less  than  half 
during  the  SAR&FUR  Phase.  Apparently,  SAR-decoys  succeeded  in  deceiving  the  SME 
in  the  SAR-only  Phase  but  many  of  them  were  correctly  rejected  during  the  SAR&FUR 
Phase.  Nonetheless,  the  absolute  level  of  FA  Rates  remained  high  during  the  SAR&FUR 
Phase.  This  may  be  attributed  to  the  inconsistent  nature  of  FUR  images,  i.  e.,  SAR- 
decoys  were  always  cold;  Targets,  however,  could  also  be  cold. 

Similar-decoys’  FA  Rates  decreased  less  between  the  Phases  than  did  all  other  decoys 
(except  FUR-decoys).  In  addition,  Similar-decoys  created  the  highest  rates  of  FA, 
indicating  that,  on  the  average,  they  were  most  similar  to  the  targets  in  both  sensor 
modalities  (SAR  and  FUR).  Target-decoys  may  have  been  similar,  in  this  respect. 
However,  some  of  the  target  pairs  were  quite  different  from  eachother  and  created  low  FA 
Rates,  thereby  reducing  the  mean  FA  Rates  of  Target-decoys  (see  Table  7  below). 

In  general,  it  is  apparent  that  the  SMEs  benefited  from  all  the  differentiating  aspects  of 
SAR  and  FUR  imaging.  The  differences  in  signature  (and  RCS)  enabled  them  to 
distinguish  the  SAR  and  FUR  decoys  from  the  real  targets.  The  distinction  between 
targets,  Similar-decoys  and  Target-decoys  could  not  be  based  solely  on  the  signature 
(because  all  these  objects  had  similar  signatures)  and  should  be  attributed  to  other 
features,  e.g.,  the  larger  target  display  size  and  higher  ground  resolution  of  the  FUR. 

Confidence  Ratings 

Confidence  Ratings  were  calculated  for  each  type  of  decoy  object,  per  SME  per  Phase 
(Table  5  and  Figure  12).  In  some  cases,  an  SME  did  not  designate  any  decoy  of  a  specific 
group.  As  a  result,  there  were  no  Confidence  Ratings  for  that  type  of  decoy  (e.g.,  one 
SME  did  not  designate  any  FUR-decoys  in  the  SAR-only  Phase,  three  subjects  did  not 
designate  Terrain-objects  in  the  SAR&FUR  Phase).  In  these  cases  the  mean  Confidence 
Score  for  the  specific  decoy  was  assigned  to  the  empty  cell. 


Table  5  -  Confidence  Rating  Confidence  Ratings  per  decoy  type,  by  Phase 

SAR-only  Phase 

SAR&FUR  Phase 

Mean 

Mean 

(Standard  Error) 

(Standard  Error) 

Confidence  Ratings 

Similar-decoy  Rates 

2.97 

5.28 

(0.51) 

(0.25) 

Target-decoy  Rates 

2.99 

4.35 

(0.48) 

(0.50) 

SAR-decoy  Rates 

3.14 

4.71 

(0.51) 

(0.34) 

FUR-decoy  Rates 

2.70 

5.12 

(0.45) 

(0.42) 
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Terrain-objects  Rates 


1.92 

(0.39) 


2.00 

(0.38) 


Figure  12  -  Confidence  Ratings  per  decoy  type,  by  Phase 

In  an  analysis  of  variance,  in  which  one  factor  was  Phase  and  the  other  Decoy  Type,  a 
main  effect  was  found  for  Phase  (F(4,70)=32.17,  p<0.001).  Averaged  across  Decoy  type, 
the  FA  Confidence  Score  was  higher  in  the  SAR&FLIR  Phase  (M=4.29,  Se=0.30)  than  in 
the  SAR-only  Phase  (M=2.74,  Se=0.43).  There  was  also  a  main  effect  for  Decoy  Type 
(F(4,70)=8.441,  p<0.001).  Averaged  across  the  two  Phases,  the  Confidence  Score  for 
Similar-decoys  (M=4.13,  Se=0.35)  was  higher  than  the  Confidence  Score  for  Target- 
decoys  (M=3.67,  Se=0.47)  (t(7)=2.32,  p<0.05,  one-tailed).  The  Confidence  Score  for 
Terrain-objects  (M=1.96,  Se=0.37)  was  lower  than  the  Confidence  Scores  for  all  other 
decoys  (Msimiiar=4.13,  Sesimiiai^O-SS  (t(7)=10.23,  p<0.001);  MDifferent=3.67,  SeDifferent=0.47 
(t(7)=6.28,  p<0.001);  MSD=3.92,  SeSD=0.41  (t(7)=9.60,  p<0.001);  Mfd=3.91,  SeFD=0.35 
(t(7)=9.55,  p<0.001)). 

Confidence  Ratings  for  all  types  of  decoys  (except  terrain-objects)  were  significantly 
higher  after  the  SAR&FLIR  Phase  than  after  the  SAR-only  Phase.  Apparently  the  SMEs 
decided  that  if  they  selected  an  object  for  the  second  time  (first  on  SAR  and  then  on 
FLJR),  it  should  have  a  higher  probability  of  being  a  target.  The  Confidence  Ratings 
seem  to  have  some  correlation  with  FA  Rates,  i.  e.,  decoys  that  produced  high  FA  Rates 
tended  to  have  high  Confidence  Ratings.  In  particular,  the  Similar-decoy  that  was  the 
most  difficult  decoy,  was  also  associated  with  the  highest  Confidence  Scores. 

Apparently,  these  decoys  were  so  similar  to  the  targets  that  the  SMEs  felt  very  confident 
that  they  had  selected  a  real  target  and  not  a  decoy. 

Overall,  the  Phase  effect  seems  much  larger  than  the  decoy  effect.  The  differences 
between  the  Confidence  Ratings  of  the  various  decoys  were  fairly  small,  except  for 
terrain-objects  which  received  much  lower  ratings. 
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Individual  Targets 


Because  of  the  relatively  low  incidence  of  each  target,  the  data  in  this  independent 
variable  are  descriptive  and  were  not  analyzed  statistically. 

Hit  and  FA  Rates  for  different  targets 

Table  6,  Figure  13  and  Figure  14  present  Hit  and  FA  Rates  of  each  target  during  the  SAR- 
only  and  the  SAR&FUR  Phases.  Corrected  FA  Scores  by  decoy  and  Phase  were 
calculated  separately  for  each  target,  as  described  above  and  in  Table  4.  These  results  are 
presented  in  Table  7. 

The  Productivity  Scores  of  each  of  the  targets  were  higher  during  the  SAR&FLIR  Phase 
(M=2.31,  Se=0.18)  than  during  the  SAR-only  Phase  (M=1.20,  Se=0.05)  (Table  2).  In  the 
SAR-only  Phase,  the  Productivity  Scores  were  fairly  similar  across  all  targets  (between 
1.01  and  1.44).  In  the  SAR&FLIR  Phase,  however,  the  difference  between  scores  is 
much  larger  (between  1.23  and  6.78).  This  is  an  additional  exemplification  of  the  crude 
performance-strategy  of  the  first  Phase  opposite  the  more  specific  and  target-dependent 
performance  strategy  apparently  employed  by  the  SMEs  during  the  second  Phase  of  the 
experiment. 

The  SCUD  was  much  easier  to  recognize  than  all  other  targets.  It  had  a  100%  Hit  Rate 
during  both  Phases.  Its  final  FA  Rate  was  the  lowest  and  its  Productivity  Scores  were 
much  higher  than  those  of  all  other  targets.  The  SCUD  was  easy  to  distinguish  from  other 
targets  (and  specifically  from  its  SA-6  Target-decoy)  because  it  was  significantly  bigger 
than  all  other  targets,  whereas  the  other  targets  had  generally  comparable  sizes. 

The  SA-6  target  produced  the  second  highest  Hit  Rates  and  the  second  highest  final 
Productivity  Score.  The  reason  for  that  is  that  the  SA-6  and  the  SCUD  served  as  each 
other’s  Target-decoy  and,  as  such,  produced  negligible  FA  Rates  (Table  7),  thereby 
reducing  overall  FA  Rates  and  increasing  the  Productivity  Scores. 


Table  6  -  Hits  Rates,  FA  Rates  and  Productivity  Scores  per  target,  by  Phase 


Hit  Rates 

SAR-only 
FA  Rates 

Productivity 

Hit  Rates 

SAR&FLIR 

FA  Rates  Productivity 

Target 

M-l 

66.7 

65.2 

1.01 

61.5 

45.3 

1.75 

T-72 

80.0 

75.0 

1.15 

65.0 

59.2 

1.30 

MLRS 

67.7 

54.1 

1.28 

58.3 

43.4 

1.36 

Radar 

77.1 

76.6 

1.09 

47.2 

37.4 

1.23 

SA-6 

87.5 

80.6 

1.12 

75.0 

25.0 

3.56 

Scud 

100.0 

73.3 

1.44 

100.0 

18.3 

6.78 
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Table  7  -  Corrected  FA  Scores  per  target,  by  decoys  and  by  Phase 


Decoy 

Similar  Target 

SAR-only 
SAR  ’ 

FLIR 

Terrain 

S  AR&FUR 

Similar  Target  SAR  FLIR 

Terrain 

Target 

M-l 

0.91 

0.58 

0.78 

0.02 

0.26 

0.63 

0.38 

0.44 

0.31 

0.04 

T-72 

0.81 

0.88 

0.97 

0.16 

0.28 

0.69 

0.79 

0.41 

0.50 

0.04 

MLRS 

0.66 

0.56 

0.95 

0.02 

0.18 

0.39 

0.41 

0.54 

0.07 

0.02 

Radar 

0.80 

0.86 

0.70 

0.23 

0.37 

0.53 

0.38 

0.20 

0.21 

0.11 

SA-6 

1.00 

0.50 

0.94 

0.00 

0.44 

0.21 

0.00 

0.50 

0.06 

0.16 

Scud 

0.88 

0.88 

0.97 

0.03 

0.28 

0.31 

0.04 

0.31 

0.03 

0.00 

The  MLRS  had  the  lowest  final  Productivity  Scores.  It  can  be  seen  that  its  Hit  Rates  as 
well  as  FA  Rates  decreased  very  little  between  Phases.  The  reasons  for  that  are  not  quite 
clear. 


The  M-l  had  a  high  final  Productivity  Score  but  the  T-72  had  a  low  score.  These  two 
tanks  served  as  each  other’s  Target-decoy.  An  interesting  asymmetry  can  be  observed  in 
Table  7.  In  the  S  AR&FUR  Phase,  the  T-72  had  a  very  high  rate  of  Target-decoy  FA,  i. 
e.,  the  M-l  was  easily  mistaken  for  a  T-72.  However,  the  Target-decoy  FA  Rate  of  the 
M-l  were  less  than  half  as  large,  i.  e.,  the  T-72  was  not  mistaken  for  a  M-l.  At  present 
we  have  no  explanation  for  this  phenomenon. 


Figure  13  -  Hit  Rates  per  decoy  type,  by  Phase 
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Figure  14  -  False  Alarm  Rates  per  Decoy  Type,  by  Phase 


Individual  Difference  Case  Explorations 

The  SMEs  who  participated  in  the  present  study  were  assigned  to  it  by  their  superior 
officers.  They  varied  in  experience,  skill  and,  perhaps,  personal  interest  and  motivation  to 
participate  in  the  experiment.  Even  though  all  the4  SMEs  had  professional  experience  as 
imagery  interpreters,  the  experimental  setup  was  new  and  unfamiliar.  Some  SMEs  had 
only  limited  SAR  experience  and  most  of  them  had  little  or  no  FUR  experience.  In 
addition,  the  fidelity  of  the  simulated  SAR  and  FUR  images  was  limited  and  their  nature 
was  unfamiliar.  The  experimental  procedure  had  only  partial  resemblance  to  the  SME’s 
familiar  working  routine:  the  background  terrain  area  was  completely  unfamiliar  and 
changed  between  trials;  recognition  was  based  solely  on  the  appearance  and  signature  of 
the  objects  and  could  not  benefit  from  additional  information  such  as  deployments  of 
objects,  environmental  effects  (e.g.,  track  scarring),  etc.  Although  briefing  and  practice 
times  were  not  limited,  it  is  quite  clear,  however,  that  within  the  45  to  75  minutes  taken 
for  preparation  the  SME  could  not  reach  a  very  high  professional  level. 

It  is  therefore  argued  that  the  results  of  the  present  study  represent  a  low  end  of 
operational  fidelity  and  that  the  potential  benefits  of  the  dual-sensor  procedure  may  be 
much  larger  than  those  indicated  by  the  current  results.  To  explore  some  aspects  of  this 
argument,  the  performance  of  individual  SMEs  was  examined  in  greater  detail.  The 
rationale  is  that  if  any  one  SME  did  much  better  than  average,  then  it  is  potentially 
possible  (through  specific  selection  criteria,  additional  training,  etc.)  for  others  to  perform 
at  least  as  well. 
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SME  #6 


SME  #6  had  no  apparent  outstanding  characteristics.  She  had  one  year  of  operational 
experience  in  image  interpretation,  little  SAR-interpretation  experience,  and  no  FUR 
experience.  Nonetheless,  she  performed  considerably  better  than  average,  and  succeeded 
in  markedly  reducing  her  FA  Rate  during  the  SAR&FUR  Phase  without  reducing  the  Hit 
Rate  (Figure  15,  Figure  16,  Figure  17).  This  kind  of  performance  indicates  high 
sensitivity  to  the  differences  between  targets  and  decoys  during  the  SAR&FUR  Phase. 
Interestingly,  the  level-of-confidence  of  SME  #6  increased  for  both  Hits  and  FA  to  the 
same  extent  from  the  SAR-only  to  the  SAR&FUR  Phase.  Apparently,  the  SME 
exhausted  her  ability  to  distinguish  decoys  from  iargets,  and  on  the  average  she  made  all 
her  decisions  at  the  same  level  of  confidence  (Figure  18,  Figure  19). 


Figure  16  -  SME  #6:  The  Difference  between  SAR-only  and  SAR&FUR 
designation  rates  for  Hits  and  False  Alarms 
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Figure  19  -  SME  #6:  The  Difference  between  SAR-only  and  SAR&MJR 
Confidence  Ratings  for  Hits  and  False  Alarms 


SME#8 


SME  #8  had  three  years  of  image  interpretation  experience,  approximately  two  years  of 
SAR  interpretation  experience,  and  some  FUR  experience.  Nevertheless,  he  performed 
rather  poorly.  During  the  SAR-only  Phase  his  Hit  and  FA  Rates  were  nearly  identical. 
During  the  SAR&FUR  Phase,  both  Hits  and  FA  reduced  nearly  to  the  same  extent 
(Figure  20,  Figure  21,  Figure  22).  This  SME  changed  his  criterion  between  the  first  and 
the  second  Phases.  However,  his  sensitivity  to  the  differences  between  targets  and  decoys 
was  very  low  during  both  Phases.  This  did  not  prevent  the  considerable  increase  in 
Confidence  Scores  between  the  first  and  the  second  Phase  (Figure  23,  Figure  24). 

Clearly,  this  kind  of  performance  has  negative  effects  on  overall  average  achievements. 
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Figure  21  -  SME  #8:  The  Difference  between  SAR-only  and  SAR&FUR 
designation  rates  for  Hits  and  False  Alarms 
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Figure  22  -  SME  #8:  Productivity  Score  by  Phase 


Figure  23  -  SME  #8:  Confidence  Ratings  by  Phases 


Figure  24  -  SME  #8:  The  Difference  between  SAR-only  and  SAR&FLIR  Confidence 

Ratings  for  Hits  and  False  Alarms 


Criterion  and  Sensitivity 

According  to  Signal  Detection  Theory,  SDT  (Tanner  and  Swets,  1954),  differences  in 
recognition  performance  can  be  related  to  criterion  as  well  as  to  sensitivity  changes.  In 
the  present  research  the  data  suggest  that  the  change  in  performance  between  Phases 
results  from  both  criterion  and  sensitivity  shifts.  The  changes  in  criterion  are  towards 
more  cautious  and  conservative  decisions,  which  cause  the  average  drop  in  both  Hits  and 
FA.  Sensitivity  in  the  SAR&FUR  Phase  is  higher  then  in  the  SAR-only  Phase,  enabling 
improved  overall  Difference  Scores  and  Productivity  Scores. 

SDT  was  designed  to  analyze  detection  of  signals  with  a  known  (or  assumed)  distribution 
against  a  background  of  noise  with  a  known  (or  assumed)  distribution.  Similar  methods 
have  been  developed  to  deal  with  non-parametric  stimuli.  (Schacter,  Israel  and  Racine, 
1999;  Snodgrass  and  Corwin,  1988).  The  descriptive  aspects  of  these  tools  can  be 
employed  to  present  an  overall  picture  of  criterion  changes,  sensitivity  and  individual 
differences.  Figure  25  presents  mean  Hit  Rates  as  a  function  of  mean  FA  Rates  for  each 
SME  in  each  Phase.  The  straight  lines  delineate  non-parametric  equivalents  of  SDT 
Receiver  Operating  Characteristic  curves,  for  the  SAR-only  Phase.  The  full  square  in  the 
center  represents  the  overall  mean  for  that  Phase.  This  point  is  connected  to  the  (0,0) 
comer  (bottom  left  -  “designate  nothing”  point)  and  to  the  (1,1)  comer  (top  right  - 
“designate  all”).  Each  of  these  two  lines  is  extrapolated  to  its  full  length.  These  lines 
depict  the  assumption  that  the  Hit  /  FA  Rates  that  are  delineated  by  them  could  be  reached 
by  adapting  a  more  conservative  criterion  (lower  left  section)  or  a  more  liberal  criterion 
(upper  right  section).  However,  performance  above  these  areas  (upper  left)  could  only  be 
reached  through  improved  sensitivity. 
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Figure  25  -Hit  Rates  as  a  function  of  FA  Rates  for  each  SME  in  each  Phase 

Figure  25  depicts  the  mean  changes  in  criterion  between  the  SAR-only  and  the 
SAR&FLIR  Phases.  It  can  be  seen  that  the  triangles,  (representing  the  SAR&FLIR  Phase) 
are,  on  the  average,  much  further  to  the  left  and  somewhat  below  the  squares 
(representing  the  SAR-only  Phase).  This  indicates  a  large  decrease  in  FA  along  with  a 
moderate  decrease  in  Hits. 

Figure  25  also  reveals  some  interesting  individual  differences.  Only  SME  #6  was 
capable  of  reducing  FA  without  reducing  Hits  as  well,  indicating  the  improvement  in  her 
performance  was  due  to  improved  sensitivity  only.  SME  #4  showed  a  similar  pattern  of 
performance  although  he  did  not  do  as  well  as  SME  #6. 

SME  #2’s  and  SME  #8’s  mean  performance  was  almost  identical  in  both  Phases.  They 
adapted  a  fairly  conservative  criterion  during  the  SAR-only  Phase  (producing  relatively 
few  hits  and  FAs)  and  became  very  conservative  during  the  SAR&FLIR  Phase.  Their 
location  inside  the  delineated  area,  in  the  SAR&FLIR  Phase,  indicates  that  they  changed 
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their  criteria  but  hardly  improved  their  sensitivity.  In  other  words,  they  benefited  very 
little  from  the  FLIR  imagery. 

SME  #5  shows  a  somewhat  similar  trend:  he  was  the  most  conservative  SME  during  the 
SAR-only  Phase  and  remained  the  most  conservative  during  the  SAR&FUR  Phase. 
However,  a  substantial  drop  in  FA,  indicating  some  improvement  of  sensitivity, 
accompanied  his  relatively  modest  drop  in  Hits.  In  contrast,  SME  #3  adopted  a  very 
liberal  criterion  during  the  SAR-only  Phase  and  was  close  to  the  maximal  possible 
number  of  Hits  and  FAs.  During  the  SAR&FUR  Phase,  however,  his  strategy  changed 
drastically,  with  a  large  decrease  in  Hits  and  a  huge  decrease  in  FA.  Overall,  he  revealed 
a  strong  change  of  strategy  with  modest  sensitivity  improvement. 
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Section  IV 


Summary  and  Recommendations 


The  growing  availability  of  sensor  imagery  in  digital  form  allows  the  pictorial  data  to  be 
stored,  processed  and  transformed  in  various  ways  that  were  not  possible  in  the  past.  One 
possibility  is  to  synchronize  and  combine  information  from  various  sources,  thereby 
gaining  from  the  benefits  of  each  of  them,  separately  and  in  combination. 

The  present  study  explored  some  potential  benefit  of  the  combined  usage  of  SAR  and 
FUR  sensor  imagery.  SAR  images  provided  wide  area  coverage  in  which  the  targets’ 
RCS  determined  their  displayed  representation  in  relatively  low-resolution.  The  simulated 
FUR  sensor  provided  an  IR  image  of  twofold  magnification  and  twice  the  resolution  of 
the  SAR  in  a  much  smaller  immediate  field  of  view  (1/16  that  of  the  SAR  image).  The 
experimental  task  posed  a  difficult  problem:  to  distinguish  between  highly  similar 
objects,  some  of  which  were  defined  as  targets,  others  were  “real”  objects  that  were 
defined  as  non-targets  (e.  g.,  friendly  forces)  and  still  others  that  were  various  types  of 
decoys.  In  this  challenging  task,  performance  with  the  SAR&FUR  combination  was 
better,  over  all,  than  performance  with  SAR-only.  During  the  SAR-only  Phase,  the  SMEs 
were  expected  to  adapt  a  “liberal”  performance  strategy  in  which  they  designated  all 
suspected  objects,  even  at  low  confidence  levels  (i.e.,  low  probability  that  the  object  was 
a  target).  This  enabled  them  to  later  reexamine  the  object  during  the  SAR&FUR  Phase. 
In  the  SAR&FUR  Phase,  the  SMEs  changed  their  performance  criterion,  became  more 
“conservative”  and  performed  their  selections  on  the  basis  of  much  higher  confidence 
levels.  As  a  result,  both  their  Hit  rates  and  FA  rates  were  lower  during  this  phase. 
However,  their  sensitivity  to  differences  between  targets  and  decoy  also  increased, 
resulting  in  a  significantly  smaller  decrease  in  Hit  rates  than  in  FA  rates.  Overall,  the 
productivity  scores  increased  significantly  between  phases  for  all  the  targets. 

The  SMEs  benefited  from  all  sensor  imaging  aspects  that  differentiate  between  the  SAR 
and  the  FUR  phenomenologies.  Their  ability  to  reject  FUR-decoys  during  the  SAR-only 
Phase  and  to  reject  S AR-decoys  during  the  SAR&FUR  Phase  was  based  on  the  SAR 
RCS  and  FUR  signatures  respectively.  Their  ability  to  distinguish  between  the  targets, 
the  Similar-decoys  and  the  Target-decoys,  however,  could  not  be  based  on  these  signature 
differences  because  the  latter  were  also  target-object  that  had  similar  signature.  Hence, 
they  must  have  been  based  on  other  aspects  such  as  the  larger  size  of  the  target  image  on 
the  display  and  the  inherently  higher  resolution  of  the  FUR  image  (supporting  an 
enhanced  ability  to  exploit  target-internal  detail). 
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In  general,  confidence  ratings  (which  were  given  after  each  target  designation  in  both 
phases)  were  low  during  the  SAR-only  Phase  and  high  during  the  SAR&FLIR  Phase. 
This  is  another  indication  of  the  SMEs’  liberal  performance  criterion  during  the  SAR- 
only  Phase  as  compared  to  the  much  more  conservative  criterion  adopted  during  the 
SAR&FLIR  Phase  (in  which  the  final  target  designation  decisions  were  made).  The 
confidence  ratings  for  correct  targets  (Hits)  were  higher  than  for  decoys  (FA).  The  effect 
was  not  very  large  but  produced  significant  difference  scores.  Interestingly  (but  not 
surprisingly),  decoys  that  were  very  hard  to  distinguish  from  targets  and  produced  high 
FA  rates  also  produced  high  confidence  ratings. 

Large  individual  differences  between  SMEs  were  observed  in  terms  of  both  the  apparent 
decision  criteria  and  overall  performance  levels.  Some  SMEs  adapted  a  fairly 
conservative  criterion  during  the  SAR-only  Phase.  This  limited  their  ability  to  improve 
performance  during  the  SAR&FLIR  Phase  and  resulted  in  poor  final  performance.  Even 
though  all  SMEs  were  imagery  ianalysts  with  various  degrees  of  experience,  they  had 
only  moderate  S  AR  experience  (and  that  with  much  lower  resolution  imagery)  and 
practically  no  FUR  experience.  In  addition,  the  SMEs  were  assigned  to  participate  in  the 
experiment  during  a  working  day  and  some  of  them  may  not  have  been  as  highly 
motivated  as  desired. 

The  rigid  procedure  in  the  experiment,  in  which  all  objects  were  first  scanned  on  SAR- 
only  and  then  with  FUR,  was  designed  for  experimental  purposes.  This  enabled  us  to 
measure  the  relative  contributions  of  the  SAR  and  of  the  FUR.  In  operational  practice,  a 
more  flexible  procedure  would  allow  the  SME  to  move  back  and  forth  between  multiple 
sensor  images. 

In  summary,  the  present  results  reveal  some  of  the  benefits  of  multiple-sensor  imagery 
interpretation.  In  view  of  the  various  research  limitations,  we  believe  that  the  results 
manifest  only  the  most  basic  levels  of  potential  benefits  and  that  appropriate  usage  by 
trained  experts  may  yield  much  higher  performance  improvements.  Research  in  this  area 
has  just  begun  and  should  be  continued  and  extended. 
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Glossary 

CAD 

Computer  aided  design 

EM 

Electro-magnetic 

EO 

Electro-optical 

FA 

False  alarm 

FUR 

Forward  looking  infrared 

FOV 

Field  of  view 

IR 

Infrared 

M 

Meter 

M 

Mean 

MLRS 

Multiple  launch  rocket  system 

NATO 

North  Atlantic  Treaty  Organization 

RCS 

Radar  cross  section 

SAR 

Synthetic  aperture  radar 

SDT 

Signal  detection  theory 

Se 

Standard  error  of  the  mean 

SME 

Subject  matter  experts 

UAV 

Uninhabited  air  vehicle 
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Appendix  1 :  Target  and  Decoy  Images 

Displayed  in  the  following  pages  are  images  of  the  targets  and  decoys  that  were  used  in 
the  experiment.  These  images  were  shown  to  the  SME  in  the  training  phase.  Each  image 
depicts  the  target  in  the  top  panel,  the  Similar-decoy  in  the  middle  panel,  and  the  SAR 
and  FUR-decoys  in  the  bottom  panel.  Objects  are  displayed  from  five  different  viewing 
angles  (top,  front,  back,  right,  and  left). 
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Figure  27  -T-72  tank 
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Figure  30  -  SA-6  launcher 


Figure  31  -  SCUD  launcher 
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Appendix  2:  SME  Instructions 

In  the  present  experiment  you  will  be  asked  to  identify  targets  in  S  AR  and  FUR  imagery. 
For  each  trial  one  of  the  following  objects  will  be  pre-briefed  as  a  target: 

0  T-72tank 
©  M-l  tank 

©  SCUD  launcher  (type  a) 

©  SA-6  launcher  (type  a) 

©  MLRS  (type  a) 

©  Radar  (type  a) 

The  targets  will  be  located  in  an  area  of  1000x1000  m.  During  each  trial,  several  targets 
(1-6)  of  the  pre-briefed  type  may  be  present.  You  will  be  asked  to  designate  all  of  them. 

In  addition,  various  other  objects,  including  targets  (other  then  the  pre-briefed  one)  and 
various  types  of  decoys,  may  be  present. 

During  the  first  phase  of  each  trial,  the  whole  area  will  be  presented  in  a  SAR  image. 

After  the  completion  of  designation  of  all  targets  (or  after  a  specified  period  of  time),  you 
will  be  able  to  view  a  FLER  image  of  the  designated  targets  and  their  area.  Each  FUR 
image  covers  an  area  of  approximately  250x250  m.  Following  each  target  designation  (on 
SAR  or  FUR)  you  will  be  asked  to  indicate  your  level  of  confidence  on  a  scale  of  7 
grades: 

1  -  Detection:  The  object  may  be  some  type  of  target. 

2  -  Recognition:  The  object  is  possibly  a  target  of  the  specified  category  (e.g.,  a  tank) 

3  -  Recognition:  The  object  is  probably  a  target  of  the  specified  category 

4  -  Recognition:  The  object  is  definitely  a  target  of  the  specified  category 

5  -  Identification:  The  object  is  possibly  the  designated  target  (e.g.,  T-72  tank) 

6  -  Identification:  The  object  is  probably  the  designated  target 

7  -  Identification:  The  object  is  definitely  the  designated  target 

0  -  Not  a  target:  In  addition,  on  FUR,  you  may  cancel  your  previous  decision  (on  SAR) 
and  determine  that  an  object  was  not  a  target. 

During  the  first  phase  of  the  experiment  (SAR)  you  should  designate  targets  even  if  your 
confidence  level  is  low,  in  order  to  reexamine  them  in  the  FUR  image.  Your  final  goal  is 
to  end  each  trial  with  as  many  correctly  designated  targets  as  possible  and  as  few 
false  alarms  as  possible. 

Putting  the  cursor  on  the  target  and  pressing  the  left  mouse  key  designates  the  target.  A 
red  box  will  appear  around  the  designated  target.  After  designation  you  will  be  asked  to 
indicate  your  level  of  confidence  by  pressing  one  of  the  FI  -  F7  keyboard  keys.  The  red 
box  will  then  change  to  blue.  After  the  designation  of  all  targets  on  SAR  press  F9  to  view 
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the  FLIR  (this  will  also  happen  automatically  at  the  end  of  elapsed  time).  Now  the  SAR 
image  will  become  inactive  and  you  may  not  designate  additional  targets  on  SAR.  Put  the 
cursor  on  a  designated  SAR  target  and  press  the  left  mouse  key  to  view  the  FLIR  imagery 
of  that  target  (Note  that  you  may  only  select  previously  designated  SAR  targets).  On  the 
SAR  image,  the  box  surrounding  the  target  will  turn  green.  Now,  designate  the  target  on 
the  FUR  image  with  the  left  mouse  button  and  specify  your  level  of  confidence  between 
1  -  7  (F1-F7).  Note  that  now  you  may  also  select  F8  to  change  your  previous  decision  and 
indicate  that  an  object  was  not  a  target.  To  complete  working  on  a  EUR  image  press  F9. 
Note  that  you  may  not  return  twice  to  the  FUR  image  of  the  same  SAR  target  (on  the 
SAR  image,  the  box  surrounding  a  target  that  was  already  seen  in  FUR  will  turn  white). 
Also  note  that  the  designated  SAR  object  may  not  be  displayed  in  the  exact  center  of  the 
FUR  image. 

You  should  try  and  work  as  fast  as  possible,  but,  more  importantly,  try  to  be  as  accurate 
as  possible.  Remember  that  the  final  goal  is  to  correctly  designate  as  many  targets  as 
possible  while  designating  as  few  false  alarms  as  possible. 

Now  you  will  perform  a  few  learning  and  practice  trials  followed  by  10  experimental 
trials. 
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